A Case Study on Bagging, Boosting, and Basic Ensembles of Neural Networks for OCR

W e study the effectiveness of three neural network ensembles in improving OCR performance: ( i ) Basic, (ii) Bagging, and (iii) Boosting. Three random character degradation models are introduced in training indivadual networks in order to reduce error correlation between individual networks and to improve the generalization ability of neural networks. We compare the recognition accuracies of these three ensembles at various reject rates. An interesting discovery in our comparison i s that although the Boosting ensemble is slightly more accurate than the Basic ensemble and Bagging ensemble at zero reject rate, the advantage of the Boosting training over the Basic and Baggang ensembles quickly disappears as more patterns are rejected, Eventually the Basic and Bagging ensembles outperform the Boosting ensemble at high reject rates. Explanation of such a phenomenon is provided in the paper. W e also apply the optimal linear combiner (in the least square error sense) to each of the three ensembles to capture dafferent error correlatzon characteristics of the three ensembles. W e find that the optimal linear combiner is very effective in reducang mean square error, but is not necessarily as effective as a simply average method in reducing classijcation error.